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This  paper  will  explore  the  feasibility  of  using  traditional 
productivity  measurement  techniques  to  measure  the  productivity  of 
science  and  technology  (S«&T)  contracting  offices.  The  primary  purpose  of 
S&T  contracting  is  to  acquire  research,  and  related  goods  and  services,  that 
are  not  directly  associated  with  the  engineering  development  of  a  specific 
system.  Most  S&T  contracting  is  done  by  offices  that  are  associated  with 
the  research  laboratories  of  the  Armed  Services.  S&T  contracts  are  usually 
funded  under  accounts  6.1  (basic  research),  6.2  (applied  research),  and 
6.3A  (large  scale  experimentation)  of  the  DOD  Research,  Development,  Test, 
and  Evaluation  budget.  Additionally,  there  is  contracting  for  specially 
funded  programs  like  manufacturing  technology  research. 


THE  PROBLEM 


Why  should  anyone  be  concerned  about  the  productivity  of  S&T 
contracting  offices?  Two  factors  drive  this  concern.  First,  the  current 
national  security  strategy  of  the  United  States  puts  unprecedented 
importance  on  the  research  work  of  the  Service  laboratories.  Second,  the 
Service  laboratories  are  making  significant  efforts  to  streamline  their 
acquisition  procedures. 

Consider  the  following  statements  from  the  1992  National  Military 
Strategy  of  the  United  States; 

"The  United  States  must  continue  to  rely  on  technological 
superiority  to  offset  quantitative  advantages,  to  minimize  risk  to 
U.  S.  forces,  and  to  enhance  the  potential  for  swift,  decisive 
termination  of  conflict."  (1) 

"Beyond  the  requirement  for  reconstitution  capability,  is  the 
compelling  need  for  continued  and  significant  R&D  in  a  wide 
spectrum  of  technologies,  applications,  and  systems."  (2) 

"Since  we  currently  have  the  most  technologically  advanced 
systems  in  the  world,  our  future  investment  choices  may  require 
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a  different  acquisition  strategy  than  we  have  followed  in  the  past. 

For  example,  full  scale  production  may  not  always  follow 
prototyping."  (3) 

These  statements  indicate  that,  for  the  foreseeable  future,  research 
will  play  a  greater  role  than  production  in  assuring  U.  S.  technological 
superiority.  This  focus  on  research  will  put  the  laboratories  in  an 
unprecedented  position  of  importance. 

This  new  focus  on  the  importance  of  research  is  occurring  at  the 
same  time  that  the  laboratories  are  engaged  in  extraordinary  efforts  to 
streamline  their  acquisition  procedures.  These  efforts  had  their  genesis  in 
widespread  criticism  from  sources  like  the  1983  White  House  Science 
Council  (the  so-called  Packard  Report);  the  Report  of  the  Carnegie 
Commission  on  Science,  Technology,  and  Government;  and  the  House 
Armed  Services  Committee  Workshop  on  Challenges  Confronting  the  DOD 
Laboratories.  Each  of  these  sources  identified  inefficient  acquisition 
procedures  as  a  problem  hindering  the  effectiveness  of  the  laboratories. 

The  FY  1990  Defense  Authorization  Act  created  the  Laboratory 
Demonstration  Program.  Congress  established  this  program  to  encourage 
DOD  to  experiment  with  liberal  new  laboratory  management  policies  in 
many  areas  including  acquisition.  The  Services  have  responded  to  this 
program  with  initiatives  to  streamline  contracting  procedures. 

Unfortunately,  it  is  difficult  to  assess  the  impact  of  these 
streamlining  initiatives.  Contracting  lead  time  might  seem  like  an  obvious 
way  to  measure  this  impact.  But,  a  reduction  in  contracting  lead  time 
would  not  be  a  measure  of  improvement  if  the  reduction  was  achieved  by 
increasing  the  number  of  people  working  in  the  contracting  office. 

Similarly,  a  decrease  in  contracting  lead  time  that  resulted  from  a  decrease 
in  workload  would  not  be  a  measure  of  improvement.  What  is  needed  is  a 
true  productivity  measurement  technique.  Such  a  technique  should  take 
into  account  the  amount  of  work  that  is  accomplished,  the  time  required  to 
accomplish  the  work,  and  the  amount  of  labor  that  went  into  the  work. 
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PREVIOUS  RESEARCH 


There  is  no  standard  way  of  measuring  the  productivity  of  a 
government  contracting  office.  A  literature  search  revealed  only  three 
previous  studies  that  addressed  the  subject.  Two  of  the  studies  focused  on 
S&T  contracting.  A  1974  study  by  Richard  B.  Kennah  suggested  using  a 
workload  measurement  system  as  a  basis  for  measuring  the  productivity 
of  what  is  now  the  Air  Force's  Wright  Laboratory  contracting  office.  (4)  In 
1983,  Christopher  D.  Miller  suggested  using  a  computerized  simulation  of 
the  S&T  contracting  process  for  the  same  purpose.  (5)  Productivity 
measurement  was  not  the  purpose  of  either  study  so  neither  author 
developed  a  productivity  measurement  technique. 

A  1988  study  by  Steven  Barclift  and  Desiree  Linson  evaluated  the 
productivity  of  an  Army  contracting  offic.j  that  was  not  engaged  in  S&T 
contracting.  (6)  These  authors  satisfied  their  research  objectives  with 
empirical  analysis.  They  made  no  attempt  to  develop  a  productivity 
measurement  technique. 

The  literature  search  did  not  reveal  any  previous  research  that 
would  be  helpful  in  developing  an  S&T  contracting  productivity 
measurement  technique.  Therefore,  the  development  of  such  a  technique 
must  rely  on  text  books  and  studies  that  were  written  for  activities  other 
than  contracting. 


WHAT  IS  PRODUCTIVITY? 

In  order  to  develop  a  measure  of  productivity,  it  is  necessary  to 
understand  what  productivity  is.  In  his  book,  the  Productivity 
Improvement  Manual.  Alan  Lawlor  gives  the  following  definition  of 
Productivity: 

"At  its  simplest,  productivity  is  the  relationship  between  goods 
produced  and  sold  or  services  provided  -  the  output,  and  the 
resources  consumed  in  doing  it  -  the  input."  (7) 
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Given  this  definition,  the  basic  math  of  productivity  measurement  is 
very  straight  forward.  It  is  simply  output  divided  by  input.  The  challenge 
of  productivity  measurement  is  to  find  measures  of  output  and  input  that 
are  both  valid  and  relatively  easy  to  obtain. 

The  primary  reason  that  an  organization  measures  productivity  is  to 
gauge  the  efficiency  of  the  organization.  Alan  Lawlor  points  out  four  other 
uses  of  productivity  measurement.  These  are  determining  the 
organization's  success  in  achieving  objectives,  effectiveness  in  maximizing 
the  use  of  resources,  comparability  with  other  organizations,  and 
productivity  trends  over  time.  (8) 

Problems  arise  if  an  organization  attempts  to  use  productivity 
measurement  in  an  inappropriate  way.  Productivity  measurement  is  not  a 
substitute  for  cost  accounting.  It  may  use  cost  data,  but  it  does  not  define 
the  cost  of  producing  goods  or  services.  Productivity  measurement  is  not  a 
substitute  for  work  measurement.  It  may  use  work  measurement  data, 
but  it  does  not  define  the  amount  of  labor  that  goes  into  a  product. 
Additionally,  productivity  measurement  does  not  identify  the  causes  of 
problems.  It  may  indicate  that  there  is  a  problem,  but  other  types  of 
analysis  are  usually  required  to  determine  the  cause  of  the  problem.  A 
good  productivity  measurement  technique  should  focus  on  productivity 
and  not  on  other  issues. 


PRELIMINARY  QUESTIONS 

Before  developing  a  productivity  measurement  technique,  two 
preliminary  questions  must  be  answered.  First,  can  traditional 
productivity  measurement  techniques  be  used?  Second,  should  total  or 
partial  productivity  measures  be  used? 
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Traditionally,  productivity  measurement  is  associated  with 
manufacturing  operations.  Manufacturing  operations  are  ideally  suited  for 
productivity  measurement.  There  are  usually  a  clearly  defined  set  of 
products  and  a  well  understood  set  of  processes  required  to  build  those 
products.  These  processes  are  often  measured  in  great  detail.  As  a  result, 
records  are  usually  available  to  show  the  resources  that  are  consumed  in 
the  manufacture  of  products. 

The  growth  of  the  service  sector  of  the  economy  has  created  new 
challenges  in  productivity  measurement.  Service  organizations  may 
produce  no  tangible  products,  have  no  consistent  work  processes,  and  lack 
the  ability  to  meaningfully  measure  resource  consumption  associated  with 
providing  specific  services.  Many  attempts  have  been  made  to  solve  the 
problem  of  service  sector  productivity  measurement.  Organizations  like 
General  Electric,  IBM,  and  the  American  Productivity  Center  have 
developed  approaches  to  the  problem.  (9)  In  their  previously  mentioned 
study,  Barclift  and  Linson  referenced  ten  different  approaches  to  service 
sector  productivity  measurement.  (10)  Unfortunately,  no  one  has  been 
able  to  develop  an  approach  that  has  gained  widespread  acceptance. 

In  his  book,  A  Practical  Guide  to  Productivity  Measurement.  Leon 
Greenberg  states  that  productivity  measurement  is  not  equally  difficult  for 
all  types  of  service  organizations.  He  points  out  that  there  are 
organizations,  like  barber  shops,  that  produce  highly  tangible  units  of 
service.  Speaking  of  such  organizations,  he  states: 

"Once  the  units  of  service  are  identified,  the  procedures  for 
measurement  are  exactly  the  same  as  for  a  plant  which 
manufactures  commodities."  (11) 

An  S&T  contracting  office  produces  tangible  units  of  service.  The 
output  of  an  S&T  contracting  office  is  a  set  of  types  of  contractual  actions. 
These  contractual  actions  result  in  a  physical  paper  product,  have  a  specific 
set  of  processes  associated  with  their  production,  and  the  time  consumed 
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by  these  processes  is  captured  by  a  management  information  system. 

Given  these  conditions,  it  is  appropriate  to  use  traditional  productivity 
measurement  techniques  to  measure  S&T  contracting  offices. 

Total  vs.  Partial  Measurement 

One  basic  decision  that  must  be  made  in  developing  a  productivity 
measurement  technique  is  determining  the  amount  of  information  that  is  to 
be  captured  by  the  technique.  In  their  book,  Productivity  by  Objectives. 
James  L.  Riggs  and  Glenn  H.  Felix  say  that  there  are  two  possible 
approaches  to  the  problem.  Ore  is  to  develop  a  total  productivity  measure 
that  relates  output  to  all  associated  if  puts.  The  other  approach  is  to 
develop  a  partial  productivity  measure  that  relates  output  to  one  type  of 
input. 


At  first  glance,  total  productivity  measures  are  intellectually 
appealing.  They  appear  to  offer  a  more  complete  picture  of  the 
productivity  of  an  organization.  However,  they  are  highly  complex  forms  of 
measurement.  They  require  the  organization  to  measure  every  type  of 
relevant  input  -  labor,  capital,  materials,  energy,  and  purchased  services. 
Measuring  all  this  input  can  be  an  overwhelming  task.  Additionally,  the 
measurement  technique  can  be  so  complex  that  it  lacks  intuitive  meaning 
to  the  people  in  the  organization.  Riggs  and  Felix  conclude: 

"Calculating  total  productivity  measure  is  an  exhausting  exercise. 

Judging  from  the  survey  reported  earlier,  not  many  companies 
undertake  the  exercise.  Bypassing  momentarily  all  the  number 
crunching  snags,  the  naked  value  of  a  total  measure  is  still 
questionable."  (12) 

Partial  productivity  measures  offer  the  possibility  of  obtaining  a 
meaningful  picture  of  organizational  productivity  in  a  way  that  is 
affordable.  These  measures  may  be  thought  of  as  being  indicators,  rather 
than  complete  measures,  of  productivity.  Nevertheless,  they  are  valuable 
tools.  Riggs  and  Felix  see  great  value  in  partial  measures.  They  state; 
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"Well  selected  productivity  indicators  or  partial  measures  furnish 
comparable  information  at  less  expense;  expensive  calipers  are 
wasted  where  a  cheap  ruler  is  sufficient."  (13) 

It  would  be  extremely  difficult  to  develop  a  total  productivity 
measure  for  an  S&T  contracting  office.  Government  accounting  procedures 
do  not  capture  the  cost  of  all  the  resources  consumed  by  an  S&T  contracting 
office.  There  are  no  allocation  procedures  that  capture  the  cost  of  the 
management  or  support  structures  in  which  the  organization  exists.  Also, 
there  are  no  depreciation  techniques  that  allow  the  reasonable  costing  of 
capital  expenditures.  Given  these  limitations,  the  development  of  a  partial 
productivity  measure  is  the  only  viable  alternative. 


SURROGATES  AND  EQUIVALENTS 

Key  concepts  in  productivity  measurement  are  the  use  of  surrogates 
and  equivalents.  These  concepts  are  particularly  important  in  the  use  of 
partial  productivity  measures.  The  use  of  a  surrogate  simply  means  the 
use  of  a  measure  of  output  or  input  that  approximates  a  measure  that  is  too 
difficult  to  obtain.  The  use  of  equivalents  means  finding  a  measure  that 
mathematically  relates  dissimilar  factors  that  appear  in  either  the  output  or 
input  parts  of  the  productivity  equation. 

The  need  for  surrogates  and  equivalents  becomes  apparent  when  one 
considers  that  most  organizations  produce  more  than  one  product.  If  an 
organization  produces  a  range  of  dissimilar  products,  it  does  little  good 
simply  to  add  up  the  total  number  of  products  produced  and  call  the  result 
output.  There  would  be  no  understanding  of  the  relative  importance  of 
each  product. 

Leon  Greenberg  suggests  that  a  good  solution  to  this  problem  is  to 
define  products  in  terms  of  the  amount  of  labor  associated  with  making  the 
product.  (14)  The  mathematical  relationship  between  the  amounts  of  labor 
associated  with  each  product  can  then  be  used  as  a  basis  for  flaking  the 
products  equivalent.  For  example,  suppose  that  a  plant  produces  two 
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products.  Widget  A  and  Widget  B.  Also,  suppose  that,  on  average,  it  takes 
one  hour  of  labor  to  build  Widget  A  and  ten  hours  to  build  Widget  B.  In 
this  case,  one  unit  of  Widget  B  would  be  the  equivalent  of  ten  units  of 
Widget  A.  This  relationship  could  be  used  to  define  the  total  output  of  the 
plant  in  terms  of  equivalent  units.  If  the  plant  produced  one  Widget  A  and 
one  Widget  B,  the  total  output  would  be  eleven  equivalent  units. 

The  same  approach  can  be  used  to  measure  input.  If  more  than  one 
category  of  worker  is  involved  in  producing  a  product,  then  the  differences 
in  cost  of  each  category  of  labor  can  be  used  as  a  basis  for  equivalence. 

Input  can  then  be  measured  in  terms  of  equivalent  units  of  labor.  It  should 
be  noted  that  in  using  labor  as  a  measure  of  input,  the  organization  is 
trying  to  capture  all  the  labor  that  is  available  for  use.  In  using  labor  as  a 
measure  of  output,  the  organization  is  trying  to  define  products  in  terms  of 
the  standard  amount  of  labor  that  it  takes  to  produce  one  unit  of  each 
product. 


DEVELOPING  A  MEASUREMENT  TECHNIQUE 

Using  the  concepts  discussed  above,  it  is  possible  to  develop  a 
productivity  measurement  technique  for  S&T  contracting  offices.  The  key 
to  developing  such  a  measure  is  finding  valid  equivalent  measures  of 
output  and  input. 

Measuring  Output 

As  previously  discussed,  S&T  contracting  offices  produce  a  clearly 
defined  set  of  products  called  contractual  actions.  These  products  are 
dissimilar.  They  vary  in  size,  complexity,  and  the  amount  of  work  required 
for  each.  There  is  no  work  measurement  system  that  measures,  or 
approximates,  the  number  of  hours  of  labor  that  goes  into  each  type  of 
action.  Therefore,  labor  hour  content  cannot  be  used  to  as  a  measure  of 
output. 
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While  S&T  contracting  offices  do  not  capture  the  labor  hours 
associated  with  contractual  actions,  their  management  information  systems 
do  capture  the  number  of  days  that  the  contracting  office  spends  working 
on  each  action.  The  average  number  of  work  days  for  each  type  of  action 
can  be  used  as  a  reasonable  surrogate  for  the  standard  labor  content  of 
each  action.  The  mathematical  relationship  between  these  standards  can  be 
used  to  develop  equivalence  relationships.  The  output  of  an  S&T 
contracting  office  would  be  stated  in  terms  of  the  number  of  equivalent 
contractual  actions  it  produced. 

Measuring  Input 

S&T  contracting  offices  know  the  total  number  of  employees  assigned 
to  the  office.  Therefore,  they  can  use  total  available  labor  as  a  measure  of 
the  input  of  the  organization.  The  size  of  the  S&T  contracting  office  work 
force  could  be  stated  in  terms  of  the  total  hours  of  labor  available. 

However,  given  that  the  labor  hour  content  for  each  type  of  contractual 
action  is  not  known,  there  would  seem  to  be  little  benefit  in  defining  input 
labor  in  terms  of  hours.  It  appears  sufficient  to  state  input  in  terms  of  the 
total  number  of  employees  in  the  organization. 

The  employees  of  S&T  contracting  offices  comprise  a  wide  range  of 
military  ranks  and  civilian  pay  grades.  It  is  necessary  to  make  these 
employees  equivalent  if  they  are  to  be  used  as  a  measure  of  input.  This 
can  easily  be  done  by  using  the  relative  difference  in  the  salaries  of  the 
different  ranks  and  grades  as  a  basis  for  equivalence.  Using  this  approach, 
an  employee  with  a  higher  salary  would  be  measured  as  the  equivalent  of 
multiple  employees  with  lower  salaries.  The  final  product  of  this  approach 
would  be  measuring  the  input  of  an  S&T  contracting  office  as  the  total 
number  of  equivalent  employees. 

Accounting  for  Variance 

It  is  extremely  important  that  S&T  contracting  offices  accomplish 
their  work  in  a  timely  manner.  The  laboratories  have  been  repeatedly 
criticized  for  slow  acquisitions.  The  discussion  above  advocates  using  the 
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approximate  average  work  days  for  each  type  of  contractual  action  as  a  set 
of  standards  for  measuring  output.  If  this  approach  is  used,  then  it  is 
extremely  important  that  the  S&T  contracting  office’s  actual  performance 
not  be  worse  than  the  standards. 

Actual  performance  is  extremely  important  when  viewed  from  the 
perspective  of  the  S&T  contracting  office's  customers.  Performance 
standards  that  are  measured  in  work  days  are  essentially  promises  to 
complete  contractual  actions  in  the  standard  amount  of  time.  A  contractual 
action  that  is  completed  late  is  not  the  same  thing  as  an  action  that  is 
completed  on  time.  The  later  a  contractual  action  is,  the  worse  it  is.  On  the 
other  hand,  a  contractual  action  that  is  completed  early  is  a  very  positive 
thing. 


The  importance  of  timeliness  means  that  variance  from  standard 
times  must  be  taken  into  account  when  measuring  productivity. 
Fortunately,  this  can  be  done  in  a  straight  forward  way.  The  S&T 
contracting  office's  management  information  system  captures  actual 
performance  on  all  contractual  actions.  These  actuals  can  be  used  to 
compute  variance  on  all  actions.  This  variance  can  be  subtracted  from,  or 
added  to,  the  standards  to  adjust  the  measure  of  output  according  to 
whether  the  S&T  contracting  office's  performance  was  either  negative  or 
positive. 


A  REAL  WORLD  APPLICATION 

The  discussion  above  presented  a  theoretical  basis  for  developing  a 
measure  of  S&T  contracting  office  productivity.  The  actual  application  of 
these  concepts  is  not  difficult.  The  discussion  below  will  use  these  concepts 
to  analyze  the  productivity  of  the  Wright  Laboratory  contracting  office  in 
FY  1990  and  FY  1991.  (15)  Wright  Laboratory  is  the  Air  Force’s  largest 
laboratory.  Its  contractual  workload  is  approximately  equal  to  the 
combined  workload  of  the  Air  Force's  other  three  laboratories. 
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Output 


The  Wright  Laboratory  contracting  office  produces  five  basic  types  of 
products.  These  are:  Phase  I  Small  Business  Innovative  Research  (SBIR  I) 
awards.  Phase  II  Small  Business  Innovative  Research  (SBIR  II)  awards. 
Program  Research  and  Development  Announcement  and  Broad  Agency 
Announcement  (PRDA/BAA)  awards,  awards  using  Request  For  Proposal 
(RFP)  procedures,  and  modifications  (MODs)  to  contracts  that  have 
previously  been  awarded. 

Wright  Laboratory  has  developed  standards  for  the  amount  of  time  it 
should  take  to  accomplish  each  type  of  action.  The  standards  are  stated  in 
terms  of  the  number  of  calendar  days  that  pass  from  the  day  of  receipt  of  a 
purchase  request  to  the  day  of  contract  award.  They  are  based  on  the 
actual  historical  performance  of  top  performing  Air  Force  laboratories. 

These  standards  are;  SBIR  I  -  30  days,  SBIR  II  -  107  days,  PRDA/BAA  -  90 
days,  RFP  -  160  days,  and  MODs  -  60  days.  These  standards  are  reasonable 
surrogates  for  the  labor  content  of  contractual  actions  and  can  be  used  as 
measures  of  output. 

Since  the  standards  are  stated  quantitatively,  it  is  not  difficult  to 
make  them  equivalent.  It  is  simply  a  matter  of  selecting  one  of  the 
standards  as  a  baseline  and  dividing  all  the  other  standards  by  the  baseline 
standard.  The  result  is  a  set  of  numbers  that  defines  each  type  of 
contractual  action  in  terms  of  the  equivalence  of  each  to  the  baseline 
contractual  action.  It  makes  little  difference  which  standard  is  used  as  the 
baseline.  But,  there  is  some  intuitive  logic  in  using  the  type  of  contractual 
action  that  comprises  the  largest  part  of  the  workload  of  the  contracting 
office.  With  this  approach,  the  baseline  would  represent  the  largest  single 
block  of  work  accomplished  by  the  organization. 

Using  the  approach  described  above,  it  is  easy  to  measure  the  output 
of  the  Wright  Laboratory  contracting  office.  The  table  below  shows  that 
output  for  FY  1990.  Column  1  shows  the  type  of  contractual  action.  Column 
2  shows  the  standard  time  to  accomplish  each  action.  Column  3  shows  the 
equivalence  of  each  type  of  contractual  action  to  the  baseline  action. 


Contract  modifications  were  chosen  as  the  baseline  type  of  action  because 
they  are  the  largest  portion  of  the  contracting  office's  workload.  Column  4 
shows  the  actual  number  of  actions  of  each  type  that  were  accomplished  in 
FY  1990.  Column  5  shows  the  equivalent  number  of  actions  of  each  type 
that  were  accomplished.  Column  5  is  simply  the  actual  number  of  actions 
multiplied  by  the  equivalence  factor. 


TYPE  ACTION 

STANDARD 

EOV  FACTOR 

if  ACTIONS 

EOV  ACTIONS 

MODS 

60 

1.00 

523 

523.00 

SBIRl 

30 

.50 

92 

46.00 

SBIR  II 

107 

1.78 

35 

62.30 

PRDA/BAA 

90 

1.50 

132 

198.00 

RFP 

160 

2.66 

146 

388.36 

Totals 

928 

1,217.66 

The  table  below 

shows  output 

for  FY  1991. 

TYPE  ACTION 

STANDARD 

EOV  FACTOR 

U  ACTIONS 

EOV  ACTIONS 

MODS 

60 

1.00 

592 

592.00 

SBIRl 

30 

.50 

82 

41.00 

SBIR  II 

107 

1.78 

50 

89.00 

PRDA/BAA 

90 

1.50 

155 

232.50 

RFP 

160 

2.66 

M 

180.88 

Totals 

947 

1,135.38 

The  tables  above  tell  an  interesting  story  about  the  output  of  the 
Wright  laboratory  contracting  office  in  FYs  1990  and  1991.  The  office 
produced  19  more  actual  contractual  actions  in  FY  1991.  Yet,  there  were  78 
fewer  awards  made  using  RFPs.  Since  these  arc  the  contracting  office's 
most  complex  and  time  consuming  products,  the  number  of  equivalent 
actions  fell  by  82.28. 

Variance 

The  figures  above  do  not  present  an  accurate  picture  of  the  real 
difference  in  the  productivity  of  the  contracting  office  between  FY  1990 
and  1991.  They  do  not  show  the  difference  in  the  speed  of  contractual 
action  execution  that  the  contracting  office  achieved  in  FY  1991.  That  was 
the  first  year  of  implementation  of  the  Laboratory  Demonstration  Program. 
The  laboratory  launched  a  major  effort  to  speed  up  the  execution  of 
contractual  actions.  The  results  of  this  effort  were  impressive.  Those 
results  illustrate  why  variance  from  standards  must  be  incorporated  into 
measuring  an  S&T  contracting  office's  productivity. 

Incorporating  variance  into  the  productivity  measure  is  not  difficult. 
The  table  below  shows  how  it  can  be  done  using  W^right  Laboratory's  FY 
1990  data.  Column  1  shows  the  type  of  contractual  action.  Column  2  shows 
the  standard  work  days  for  each  type  of  action.  Column  3  shows  the  actual 
number  of  actions  of  each  type  that  were  accomplished.  Column  4  shows 
the  actual  number  of  actions  multiplied  by  the  standard  for  each  type  of 
action.  The  numbers  in  column  4  represent  the  total  number  of  work  days 
of  effort  that  would  have  been  expended  on  each  type  of  action  if  all 
actions  had  been  accomplished  in  the  standard  amount  of  time.  Column  5  is 
the  actual  number  of  workdays  that  were  expended  on  each  type  of  action^ 
The  difference  between  column  4  and  column  5  is  the  total  variance 
between  the  number  of  work  days  that  should  have  been  expended  on  the 
contracting  offices  work  load  and  the  number  of  work  days  that  were 
expended.  For  FY  1990,  this  was  a  large  negative  variance.  If  this  negative 
variance  is  divided  by  the  baseline  standard  of  60  days,  the  result  is  the 
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number  of  equivalent  actions  that  this  negative  variance  represents.  Thus, 
negative  variance  could  be  considered  negative  equivalent  actions. 


TYPE  ACTION 

STANDARD 

#  ACTIONS 

STD  TIME 

ACTUAL  TIME 

MODS 

60 

523 

31,380 

31,903 

SBIRl 

30 

92 

2,760 

9,016 

SBIRfi 

107 

35 

3,745 

3,955 

PRDA/BAA 

90 

132 

11,880 

18,480 

RFP 

160 

146 

23.360 

36.938 

Totals 

73,125 

100,292 

Variance 

-27,167 

If  the  negative  equivalent  actions  of  -27,167  days  is  divided  by  the 
baseline  standard  of  60  days,  the  result  is  452.78  negative  equivalent 
actions.  Earlier  calculations  showed  a  production  of  1,217.66  equivalent 
actions  in  FY  1990.  However,  this  number  was  not  weighted  for  variance. 

It  is  apparent  that,  in  FY  1990,  the  contracting  office  fell  behind  desired 
standards  by  more  than  a  third.  The  figure  of  452.78  negative  equivalent 
actions  reflects  this  fact.  If  these  452.78  negative  equivalent  actions  are 
subtracted  from  the  1,217.66  unweighted  equivalent  actions,  the  result  is 
764.88  weighted  equivalent  actions.  This  figure  is  a  more  realistic  measure 
of  the  output  of  the  Wright  Laboratoiy  contracting  office  in  FY  1990.  It 
reflects  serious  negative  impact  of  finishing  behind  standard  times  on 
every  category  of  contractual  action. 

The  work  day  standards  used  in  the  calculations  above  were  adopted 
by  Wright  Laboratory  late  in  FY  1990.  There  was  little  chance  that  the 
contracting  office  could  conform  to  the  standards  in  that  fiscal  year.  They 


were  established  primarily  as  a  baseline  for  measuring  future  performance. 
In  FY  1991,  the  laboratory  experienced  the  benefits  of  a  vigorous  Total 
Quality  Management  program  and  the  Laboratory  Demonstration  Program. 
There  was  a  dramatic  improvement  in  the  time  required  to  execute 
contractual  actions-  This  improvement  is  reflected  in  the  table  below. 


TYPE  ACTION 

STANDARD 

#  ACTIONS 

STD  TIME 

ACTUAL  TIME 

MODS 

60 

592 

35,520 

33,744 

SBIR  1 

30 

82 

2,460 

1,804 

SBIR  II 

107 

50 

5,350 

2,850 

PRDA/BAA 

90 

155 

13,950 

12,090 

RFP 

160 

68 

10,880 

9,384 

Totals 

68,160 

59,872 

Variance 

8,288 

As  these  calculations  demonstrate,  the  Wright  Laboratory  contracting 
office  went  from  a  substantial  negative  variance  in  FY  1990  to  a  positive 
variance  in  FY  1991,  This  positive  variance  represents  a  real  increase  in 
mission  accomplishment  for  both  the  contracting  office  and  the  laboratory 
as  a  whole.  Dividing  the  8,288  day  positive  variance  by  the  60  day 
standard  baseline  results  in  a  positive  variance  stated  as  138.13  equivalent 
actions.  Earlier  calculations  showed  a  production  of  1,135.38  unweighted 
equivalent  actions  in  FY  1991.  By  adding  the  impact  of  positive  variance, 
the  result  is  a  figure  of  1,273.51  weighted  equivalent  actions.  This  figure 
reflects  a  meaningful  increase  in  the  output  of  the  contracting  office. 

All  calculations  so  far  have  been  presented  using  tables.  Tabular 
presentations  have  been  used  because  they  make  it  easier  to  follow  the 
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development  of  the  numbers.  Actually,  it  is  possible  to  compute  total 
unweighted  equivalent  actions  without  using  the  equivalence  factors  used 
in  the  first  two  tables.  Unweighted  equivalent  actions  can  be  computed 
directly  from  the  total  standard  times  shown  in  the  third  and  fourth  tables. 
The  only  necessary  calculation  is  to  divide  the  total  standard  time  by  the 
baseline  standard  of  60  days.  For  FY  1990,  this  would  mean  dividing  the 
total  standard  time  of  73,125  days  by  60  for  a  result  of  1,218.75  equivalent 
actions.  For  FY  1991,  the  calculation  would  be  68,160  days  divided  by  60 
for  a  result  of  1,136  equivalent  actions.  The  results  are  slightly  higher  for 
the  simplified  calculations  because  of  the  greater  rounding  error  in  the 
tabular  calculations. 

Weighted  equivalent  actions  are  similarly  easy  to  calculate  directly. 
Weighted  equivalent  actions  are  equal  to: 


Total  Standard  Time  +  (Total  Standard  Time  -  Actual  Time) 


Baseline  Standard 


Applying  this  formula  to  the  FY  1990  figures  would  yield: 


73,125  +  (73,125  -  100,292) 

_  =  765.97  Weighted  Eqv  Actions 

60 


Applying  this  formula  to  the  FY  1991  figures  would  yield: 


68160  +  (68,160  -  59,872) 

_  =  1,274.13  Weighted  Eqv  Actions 


60 


The  formulas  shown  above  are  the  quickest  way  to  compute  weighted 
equivalent  actions.  Again,  the  results  are  slightly  higher  because  of  the 
rounding  error  associated  with  the  tabular  presentation.  These  formulas 
are  very  easy  to  use  in  conjunction  with  S&T  contracting  management 
information  systems.  Since  weighted  equivalent  actions  are  the  most 
desirable  measure  of  output,  these  formulas  should  be  used  to  compute 
directly  the  output  of  an  S&T  contracting  office. 

Input 

The  Wright  Laboratory  contracting  office  is  the  largest  S&T 
contracting  office  in  the  Air  Force.  In  FY  1991,  the  office  was  authorized  to 
have  185  employees.  The  payroll  cost  of  these  employees  is  many  times 
larger  than  the  contracting  office's  expenditures  for  all  other  purposes.  The 
high  relative  cost  of  employees  makes  labor  an  extremely  good  measure  of 
input. 

The  most  intuitively  logical  way  to  measure  input  labor  is  in  terms  of 
number  of  employees.  If  total  weighted  equivalent  actions  are  divided  by 
the  number  of  employees,  the  result  is  a  measure  of  productivity  that  is 
defined  as  the  number  of  equivalent  actions  per  employee.  This  definition 
appeals  to  common  sense.  Use  of  other  surrogates  for  measuring  input 
would  have  less  intuitive  appeal.  If  input  were  defined  in  terms  of  labor 
hours  or  dollars,  the  result  would  be  a  measure  of  equivalent  actions  per 
hour  or  dollar.  These  measures  would  result  in  very  small  fractional 
numbers  that  convey  little  intuitive  meaning. 

Given  that  number  of  employees  is  the  best  measure  of  input,  the 
problem  of  equivalence  must  be  resolved.  The  Wright  Laboratory 
contracting  office’s  FY  1991  personnel  authorizations  by  grade  and  rank  are 
as  follows: 


Militarv 

Civilian 

LTC  -2 

GM15  -  1 

Maj  -3 

GM14-5 

Cpt  -  10 

GM13  -13 

lit  -4 

GS13  -  15 

Msgt  -1 

GS12  -  69 

Amn  -  3_ 

GSll  -  9 

23 

GS7  -2 

GS6  -  10 

GS5  -  ^ 

162 

This  diverse  mix  of  personnel  emphasizes  the  need  for  equivalence. 
One  approach  to  equivalence  would  be  to  develop  equivalence  factors  for 
all  grades  and  ranks  of  employees.  This  could  be  done  by  using  the 
standard  salary  of  the  grade  with  the  largest  number  of  employees  as  a 
baseline  and  dividing  the  salary  of  all  other  grades  and  ranks  by  the 
baseline  salary.  By  doing  this  the  work  force  could  be  defined  in  terms  of 
an  equivalent  number  of  the  largest  group  of  employees. 

The  equivalence  factor  approach  has  drawbacks.  The  number  of 
people  that  actually  work  for  a  contracting  office  fluctuates  continuously. 
Personnel  authorizations  change  in  reaction  to  changes  in  mission  and 
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workload.  Employees  retire,  move  to  other  jobs,  and  take  extended  leaves 
to  pursue  education.  Budget  limitations  often  prevent  hiring  against  vacant 
authorized  positions.  This  personnel  turmoil  would  make  it  extremely 
difficult  to  maintain  a  system  of  equivalence  factors.  It  could  be  done,  but 
it  would  require  monthly  adjustments  to  compensate  for  work  force 
fluctuations. 

It  would  be  extremely  difficult  to  reconstruct  equivalence  factors  for 
the  Wright  Laboratory  contracting  office  for  FYs  1990  and  1991.  During 
those  two  years,  the  contracting  office  began  drawing  down  support  of  one 
major  customer  while  simultaneously  adding  a  new  geographically 
separated  branch  to  support  a  new  customer.  The  office  also  experienced 
unanticipated  military  moves  and  employees  that  were  on  extended 
absences  because  of  illnesses  or  full  time  training. 

Fortunately,  there  is  a  simple  way  to  compute  equivalent  employee 
head  count  without  using  equivalence  factors.  It  is  easy  to  obtain  actual 
expenditures  for  employee  pay.  If  total  employee  pay  is  divided  by  a 
baseline  salary,  the  result  is  the  number  of  equivalent  employees  that  were 
actually  compensated.  This  approach  results  in  an  equivalent  head  count 
that  is  both  accurate  and  easy  to  compute. 

Total  salary  costs  for  the  Wright  Laboratory  contracting  office  were 
$5,921  million  in  FY  1990  and  $7,251  in  FY  1991.  The  largest  single  pay 
grade  of  employees  during  both  years  was  GS12.  The  government  pay 
tables  provide  a  standard  salary  for  a  mid-step  GS12.  This  salary  was 
$40,640  in  FY  1990  and  $42,306  in  FY  1991.  Using  these  figures,  the 
equivalent  head  counts  for  FY  1990  and  1991  can  be  computed  as  follows: 

$5,921  Million 

FY  1990  -  _  =  145.69  Eqv  Employees 

$40,640 
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$7,251  Million 


FY  1991  -  _  =  171,39  Eqv  Employees 

$42,306 

These  numbers  of  equivalent  employees  are  a  good  surrogate  for 
measuring  labor  input.  They  are  a  valid  basis  for  computing  productivity. 

Productivity 

At  this  point,  measuring  productivity  is  a  very  straight  forward 
procedure.  It  is  simply  a  matter  of  dividing  the  measure  of  output  - 
weighted  equivalent  contractual  actions,  by  the  measure  of  input  - 
equivalent  employees.  The  calculations  are  as  follows: 


765.97 

FY  1990  -  _  =  5.26  Eqv  actions  per  Eqv  employee 

145.69 

1274.13 

FY  1991  -  _  =  7.43  Eqv  actions  per  Eqv  employee 

171.39 


It  is  also  a  straight  forward  procedure  to  calculate  the  change  in 
productivity  between  FYs  1990  and  1991.  The  calculations  are  as  follows: 


7.43  -  5.26 

_  X  100  =  41.25% 

5.26 
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The  calculations  above  provide  a  valid  measure  of  the  productivity  of 
the  Wright  Laboratory  contracting  office  in  FY  1990  and  1991.  They  take 
into  account  complex  interrelationships  between  changes  in  workload, 
employment  levels,  and  speed  of  contractual  action  execution.  They  are 
based  on  easily  obtainable  numbers  and  simple  mathematics.  They 
demonstrate  that  Wright  Laboratory  achieved  a  significant  increase  in 
contracting  office  productivity  in  FY  1991. 


SUMMARY 

This  paper  has  demonstrated  a  relatively  simple  technique  for 
measuring  the  productivity  of  S&T  contracting  offices.  Using  the 
terminology  of  Riggs  and  Felix,  this  technique  is  a  cheap  ruler  rather  than 
an  expensive  caliper.  Nevertheless,  the  technique  should  prove  sufficiently 
robust  to  accommodate  large  differences  in  work  load  and  employment 
level.  Individual  S&T  contracting  offices  should  be  able  to  adopt  the 
technique  without  too  much  difficulty.  Adopting  the  technique  at  the 
Service  or  DOD  level  would  require  development  of  Service-wide  or  DOD- 
wide  standards  for  the  number  of  days  required  to  execute  various  types  of 
S&T  contractual  actions.  Given  the  abundance  of  management  information 
system  data  that  is  available,  this  should  not  be  too  difficult  a  task. 

The  proposed  productivity  measurement  technique  does  a  good  job  of 
measuring  the  efficiency  of  S&T  contracting  offices.  The  technique  does  not 
answer  all  the  questions  that  might  be  asked  about  the  functionality  of  an 
S&T  contracting  office.  It  does  not  define  the  maximum  output  capabilities 
of  a  given  office.  Therefore,  it  cannot  be  used  to  gauge  a  contracting  office's 
productivity  relative  to  what  it  is  capable  of  doing.  The  technique  does  not 
involve  cost  data.  Therefore,  it  cannot  be  used  to  analyze  the  costs 
associated  with  executing  contractual  actions.  These  additional  questions 
may  be  matters  of  valid  concern.  But,  answering  them  will  require  the 
development  of  work  measurement  and  accounting  systems  that  do  not 
exist.  Developing  such  systems  would  be  major  undertakings  that  would 
require  large  commitments  of  time  and  resources.  The  proposed  approach 


provides  a  oasic  measure  of  productivity  that  is  intuitive,  easy  to  compute, 
and  based  on  data  that  is  available  now. 

It  is  recommended  that  the  Air  Force  incoporate  the  proposed 
productivity  measurement  technique  into  its  system  of  measuring  the 
performance  of  S&T  contracting  offices.  The  Air  Force  should  take  the  lead 
because  this  paper  has  proven  that  the  proposed  technique  is  easily 
implemented  using  the  Air  Force’s  existing  S&T  contracting  management 
information  system.  The  results  of  the  Air  Force  implementation  of  the 
technique  should  be  reviewed  by  the  OSD  team  responsible  for 
implementation  of  the  Laboratory  Demonstration  Program  contracting 
initiatives.  If  the  results  of  the  Air  Force  implementation  are  favorable, 
then  the  technique  should  be  considered  for  DOD-wide  implementation. 
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